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In an electric power system, demand fluctuations may result in significant ancillary cost to suppliers. Fur- 
thermore, in the near future, deep penetration of volatile renewable electricity generation is expected to 
exacerbate the variability of demand on conventional thermal generating units. We address this issue by 
explicitly modeling the ancillary cost associated with demand variability. We argue that a time-varying price 
equal to the suppliers' instantaneous marginal cost may not achieve social optimality, and that consumer 
demand fluctuations should be properly priced. We propose a dynamic pricing mechanism that explicitly 
encourages consumers to adapt their consumption so as to offset the variability of demand on conventional 
units. Through a dynamic game-theoretic formulation, we show that (under suitable convexity assumptions) 
the proposed pricing mechanism achieves social optimality asymptotically, as the number of consumers 
increases to infinity. Numerical results demonstrate that compared with marginal cost pricing, the proposed 
mechanism creates a stronger incentive for consumers to shift their peak load, and therefore has the potential 
to reduce the need for long-term investment in peaking plants. 
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1. Introduction 

This paper is motivated by the fact that fluctuations in the demand for electricity to be met by 
conventional thermal generating units typically result in significantly increased, and nontrivial, 
ancillary costs. Today, such demand fluctuations are mainly due to time-dependent consumer 
preferences. In addition, in the future, a certain percentage of electricity production is required 
by law in many states in the U.S. to come from renewable resources (Barbose et al. 2008). The 
dramatic volatility of renewable energy resources may aggravate the variability of the demand for 
conventional thermal generators and result in significant ancillary cost. More concretely, either 
a demand surge or a decrease in renewable generation may result in (i) higher energy costs due 
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to the deployment of peaking plants with higher ramping rates but higher marginal cost, such 
as oil/gas combustion turbines, and (ii) the redispatch cost 1 that the system will incur to meet 
reserve constraints if the increase of demand (or decrease of renewable generation) causes a reserve 
shortage. 

There is general agreement that charging real-time prices (that reflect current operating con- 
ditions) to electricity consumers has the potential of reducing supplier ancillary cost, improving 
system efficiency, and lowering volatility in wholesale prices (US Department of Energy 2006, Spees 
and Lave 2008, Chao 2010). Therefore, dynamic pricing, especially real-time marginal cost pricing, 
is often identified as a priority for the implementation of wholesale electricity markets with respon- 
sive demand (Hogan 2010), which in turn raises many new questions. For example, should prices 
for a given time interval be calculated ex ante or ex post? Does real-time pricing introduce the 
potential for new types of market instabilities? How is supplier competition affected? In this paper, 
we abstract away from almost all of these questions and focus on the specific issue of whether 
prices should also explicitly encourage consumers to adapt their demands so as to reduce supplier 
ancillary cost. 

To illustrate the issue that we focus on, we note that a basic model of electricity markets assumes 

that the cost of satisfying a given level A t of aggregate demand during period t is of the form 

C(A t ). It then follows that in a well- functioning wholesale market, the observed price should more 

or less reflect the marginal cost C'(A t ). In particular, prices should be more or less determined 

by the aggregate demand level. Empirical data do not quite support this view. Fig. 1 plots the 

real-time system load and the hourly prices on February 11, 2011 and on February 16, 2011, as 

reported by the New England ISO (ISO New England Inc. 2011). We observe that prices do not 

seem to be determined solely by A t but that the changes in demand, A t — A t _i, also play a major 

role. In particular, the largest prices seem to occur after a demand surge, and not necessarily at 

1 A certain level of reserve must always be maintained in an electric power system. Local reserve shortages are usually 
due to the quick increase of system load rather than a capacity deficiency. If the increase of system load makes the 
system short in reserves, the system will redispatch resources to increase the amount of reserves available. Redispatch 
generally increases the generation cost and results in higher prices. The redispatch cost can be very high (cf. Section 
2.3.2 of ISO New England Inc. (2010)). 
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Figure 1 Real-time prices and actual system load, ISO New England Inc. Blue bars represent the real-time system 
loads and the dots connected by a black line represent the hourly prices. 

the hour when the load is highest. We take this as evidence that the total cost over T + 1 periods 
is not of the form 

T 



t=0 



but rather of the form 



J2{C(A t ) + H(A t _ u A t )), 



(1) 



for a suitable function H. 

We take the form of Eq. (1) as our starting point and raise the question of the appropriate prices. 
A naive view would argue that at time t, A t _\ has already been realized, and taking its value for 
granted, a consumer should be charged a unit price equal to 



C{A t 



_d_ 
DA 



H(A t _ 1 ,A t ), 



(2) 



which is the supplier's marginal cost at stage t. We refer to this naive approach as "marginal 
cost pricing" (MCP). However, a simple argument based on standard mathematical programming 
optimality conditions shows that for system optimality to obtain, the demand A t _i should also 
incur (after A t is realized) a unit price of 





dA t -r 



H(A t _ 1 ,A t ). 



(3) 
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In day-ahead markets, suppliers typically carry out an intertemporal optimization, and it is rea- 
sonable to expect that the two types of marginal costs, captured by Eqs. (2) and (3), are both 
properly accounted for. However, in current real-time balancing markets, once A t _i is realized, a 
supplier will aim at charging the marginal cost in Eq. (2), but will be unable to charge the addi- 
tional marginal cost in Eq. (3) to the past demand A t _ 1 . In contrast, the pricing mechanism that 
we propose and analyze in this paper is designed to include the additional marginal cost in Eq. 
(3)- 2 

The actual model that we consider will be richer from the one discussed above in a number 
of respects. It includes an exogenous source of uncertainty (e.g., representing weather conditions) 
that has an impact on consumer utility and supplier cost, and therefore the model can incorporate 
the effects of volatile renewable electricity production. It allows for consumers with internal state 
variables (e.g., a consumer's demand may be affected by how much electricity she has already 
used). It also allows for multiple consumer types (i.e., with different utility functions and different 
internal state dynamics). Consumers are generally modeled as price-takers, as would be the case 
in a model involving an infinity (a continuum) of consumers. However, we also consider the case 
of finite consumer populations and explore certain equilibrium concepts that are well-suited to the 
case of finite but large consumer populations. On the other hand, we ignore most of the distinctions 
between ex post and ex ante prices. Instead, we assume that at each time step, the electricity 
market clears. The details of how this could happen are important, but are generic to electricity 
markets, hence not specific to our models, and somewhat orthogonal to the subject of this paper. 
(See however Appendix B for some discussion of implementation issues.) 

The ancillary cost function H(A t _i,A t ) is of course a central element of our model. How can we 
be sure that this is the right form? In general, redispatch and reserve dynamics are complicated 
and one should not expect such a function to capture all of the complexity of the true system costs; 

2 In current two-settlement systems, the real-time prices are charged only on the difference of the actual demand and 
the estimated demand at the day-ahead market. However, the two-settlement system provides the same real-time 
incentives to price-taking consumers, as if they were purchasing all of their electricity at the real-time prices (cf. 
Chapter 3-2 of Stoft (2002)). 
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perhaps, a more complex functional form such as H(A t _ 2 ,A t _ 1 ,A t ) would be more appropriate. 
We believe that the form we have chosen is a good enough approximation, at least under certain 
conditions. To argue this point, we present in Appendix A an example that involves a more detailed 
system model (in which the true cost is a complicated function of the entire history of demands) 
and show that a function of the form H(A t -i,A t ) can capture most of the cost of ancillary services. 

1.1. Summary of contributions 

Before continuing, we provide here a roadmap of the paper together with a summary of our main 
contributions. 

(a) We provide a stylized (yet quite rich) model of an electricity market, which incorporates the 
cost of ancillary services (cf. Section 2). 

(b) We provide some justification of the form of the cost function in our model, as a reasonable 
approximation of more detailed physical models (cf. Appendix A). 

(c) We propose a pricing mechanism that properly charges for the effects of consumer actions 
on ancillary services (cf. Section 3). 

(d) For a continuum model involving nonatomic price-taking consumers, we consider Dynamic 
Oblivious Equilibria (DOE), in which every consumer maximizes her expected payoff under the 
sequence of prices induced by a DOE strategy profile (Section 4). We show that (under standard 
convexity assumptions), our mechanism maximizes social welfare (cf. Theorem 2 in Section 6). 

(e) We carry out a game-theoretic analysis of the case of a large but finite number of consumers. 
We show that a large population of consumers who act according to a DOE (derived from an 
associated continuum game) results in asymptotically optimal (as the number of consumers goes to 
infinity) social welfare (cf. Theorem 2 in Section 6), and asymptotically maximizes every consumer's 
expected payoff (this is an "asymptotic Markov equilibrium" property; cf. Theorem 1 in Section 
5). 

(f) We illustrate the potential benefits of our mechanism through a simple numerical example. 
In particular, we show that compared with marginal cost pricing, the proposed mechanism reduces 
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the peak load, and therefore has the potential to reduce the need for long-term investments in 
peaking plants (cf. Appendix E). 

1.2. Related literature 

There are two streams of literature, on electricity pricing and on game theory, that are relevant to 
our work, and which we now proceed to discuss, while also highlighting the differences from the 
present work. 

Regarding electricity markets, the impact of supply friction on economic efficiency and price 
volatility has received some recent attention. Mansur (2008) shows that under ramping constraints, 
the prices faced by consumers may not necessarily equal the true supplier marginal cost. In a 
continuous-time competitive market model, Cho and Meyn (2010a) show that the limited capa- 
bility of generating units to meet real-time demand, due to relatively low ramping rates, does not 
harm social welfare, but may result in extreme price fluctuations. In a similar spirit, Kizilkale and 
Mannor (2010) construct a dynamic game-theoretic model to study the tradeoff between economic 
efficiency and price volatility. Closer to the present paper, Cho and Meyn (2010b) construct a 
dynamic newsboy model to study the reserve management problem in electricity markets, where 
the demand is assumed to be exogenous. The supplier cost in their model depends not only on 
the overall demand, but also on the generation resources used to satisfy the demand. For exam- 
ple, a quickly increasing demand may require more responsive and more expensive resources (e.g., 
peaking generation plants). 

To study the impact of pricing mechanisms on consumer behavior and load fluctuations, we 
construct a dynamic game-theoretic model that differs from existing dynamic models for electricity 
markets and incorporates both the consumers' responses to real-time price fluctuations and the 
suppliers' ancillary cost incurred by load swings. Some major differences between our model and 
existing ones are discussed at the end of Section 2. 

On the game-theoretic side, the standard solution concept for stochastic dynamic games is the 
Markov perfect equilibrium (MPE) (Fudenberg and Tirole 1991, Maskin and Tirole 1988), 
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involving strategies where an agent's action depends on the current state of all agents. As the num- 
ber of agents grows large, the computation of an MPE is often intractable (Doraszelski and Pakes 
2007). For this reason, alternative equilibrium concepts, for related games featuring a nonatomic 
continuum of agents (e.g., "oblivious equilibrium" or "stationary equilibrium" for dynamic games 
without aggregate shocks), have received much recent attention (Weintraub et al. 2009, Adlakha 
et al. 2011). 

There is a large literature on a variety of approximation properties of nonatomic equilibria (Mas- 
Colell and Vives 1993, Al-Najjar 2004, 2008). Recently, Adlakha et al. (2011) derive sufficient 
conditions for a stationary equilibrium strategy to have the Asymptotic Markov Equilibrium 
(AME) property, i.e., a stationary equilibrium strategy asymptotically maximizes every agent's 
expected payoff (given that all the other agents use the same stationary equilibrium strategy), as 
the number of agents grows large. Their model includes random shocks that are assumed to be 
idiosyncratic across agents. However, in the problem that we are interested in, it is important to 
incorporate aggregate shocks (such as weather conditions) that have a global impact on all agents. 
In this spirit, Weintraub et al. (2010) consider a market model with aggregate profit shocks, and 
study an equilibrium concept at which every firm's strategy depends on the firm's current state and 
on the recent history of the aggregate shock. For a general dynamic game model with aggregate 
shocks, Bodoh-Creed (2010) shows that a nonatomic counterpart of an MPE, which we refer to as a 
Dynamic Oblivious Equilibrium (DOE) in this paper, asymptotically approximates an MPE 
in the sense that as the number of agents increases to infinity, the actions taken in an MPE can 
be well approximated by those taken by a DOE strategy of the nonatomic limit game. However, 
without further restrictive assumptions on the agents' state transition kernel, the approximation 
property of the actions taken by a DOE strategy does not imply the AME property of the DOE, 
and we are not aware of any AME results for models that include aggregate shocks. Our work is 
different in this respect: for a dynamic nonatomic model with aggregate shocks, which is a simplified 
variation of the general model considered in Bodoh-Creed (2010), we prove the AME property of 
a DOE. 
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The efficiency of nonatomic equilibria for static games has been addressed in recent research 
(Roughgarden and Tardos 2004, Milchtaich 2004, Bodoh-Creed 2011). For a dynamic industry 
model with a continuum of identical producers and exogenous aggregate shocks, Lucas and Prescott 
(1971) show (under convexity assumptions) that the expected social welfare is maximized at a 
unique competitive equilibrium. In a similar spirit, in this paper we show (under convexity assump- 
tions) that the proposed pricing mechanism maximizes the expected social welfare in a model 
involving a continuum of (possibly heterogeneous) consumers. We also consider the case of a large 
but finite number of consumers, and show that the expected social welfare is approximately maxi- 
mized if all consumers act according to a nonatomic equilibrium (DOE). For large dynamic games, 
the asymptotic social optimality of nonatomic equilibria (DOEs) established in this paper seems 
to be new. 

2. Model 

We consider a (T + l)-stage dynamic game with the following elements: 

1. The game is played in discrete time. We index the time periods with t = 0, 1, . . . ,T. Each 
stage may represent a five minute interval in real-time balancing markets where prices and dispatch 
solutions are typically provided at five minute intervals. 

2. There are n consumers, indexed by 1, . . . , n. 

3. At each stage t, let s t £ S be an exogenous state, which evolves as a Markov chain and 
whose transitions are not affected by consumer actions. The set S is assumed to be finite. In 
electricity markets, the exogenous state may represent time and/or weather conditions, which 
impact consumer utility and supplier cost. It may also represent the level of renewable generation. 

4. For notational conciseness, for t > 1, let s t = (s t _i,s t ), and let s = s . We use S t to denote 
the set of all possible s t . We refer to s t as the global state at stage t. 

5. Given an initial global state s , the initial states (types) of the consumers, {xj,o}" = i, 
are independently drawn according to a probability measure ?7 S0 over a finite set X . We use X to 
denote the cardinality of X . 
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6. The state of consumer i at stage t is denoted by x ijt . At t = 0, consumer i's initial state, 
Xifl, indicates her type. For t = l,...,T, we have x i)t = (x it0 , z i)t ), where z ijt £ Z and Z = [0, Z] is 
a compact subset of R. The variables {^,t}" = i allow us to model intertemporal substitution effects 
in consumer i's demand. 

7. We use X t to denote a consumer's state space at stage t. In particular, at stage t>l, 
Xt = Xq x >2^. 

8. At stage t, consumer i takes an action a^ t and receives a nonnegative utility 3 Ut(xi t t,s t ,(ii,t)- 

9. Each consumer's action space is A = [0,5], where 5 is a positive real number. (In the 
electric power context, B could reflect a local transmission capacity constraint.) 

10. We use A t = Ym=\ a i,t to denote the aggregate demand at stage t. 

11. Given consumer i's current state, x iit , and the next exogenous state s t+ i, the next state of 
consumer i is determined by her action taken at stage t, i.e., x i<t+ i = (x i;0 , z iit+ i), where z iit+ i = 
r(xi tt ,(ii it ,s t+ i), for a given function r. 

12. Let G t = A t + R t be the capacity available at stage t, where R t is the system reserve at 
stage t. For simplicity, we assume that the system reserve at stage t depends only on the current 
aggregate demand, A t , and the current exogenous state s t . That is, we have R t = g(A t ,s t ) for a 
given function of g that reflects the reserve policy of the system operator. 

13. At stage t, let C(A t ,R t ,s t ) be the total conventional generation cost, that is, the sum of 
the supplier's cost to meet the aggregate demand A t through its primary energy resources, e.g., 
base- load power plants, and the cost to maintain a system reserve R t . Since R t depends only on 
A t and s t , we can write C(A t ,R t ,s t ) as a function of A t and s t , i.e., there exists a primary cost 
function C:lx5-> [0, oo) such that C(A t ,s t ) = C(A t ,R t ,s t ). We assume that for any s G S, 
C(-,s) is nondecreasing. 

14. At stage t>l, let H(A t _ 1 , A t , Rt-i, Rt, s t ) denote the ancillary cost incurred by load swings 4 . 

3 At t = 0, Uo is a mapping from Xq x <S x A to [0, oo), while for t > 1, U t is a mapping from Xo x Z x <S x A to [0, oo). 

4 In general, the supplier ancillary cost may depend on the entire history of system load and global states. However, 
ancillary cost functions with the simple form H(At~i,At,Rt-i,Rt,s t ) can serve as a good approximation of the 
supplier's true ancillary cost (cf. Appendix A) . 
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Since R t depends only on A t and s t , we can write H(A t _ 1 ,A t ,R t _ 1 ,R t ,s t ) as a function of A t ^ 1 , 
A t , s t _i, and s t , i.e., there exists an ancillary cost function H : M 2 x S 2 — > [0,oo) such that 
H = H(A t _ 1 ,A t ,s t ). The ancillary cost at stage is assumed to be a function of s and A . 
15. At stage 0, the total supplier cost is of the form 

C(A ,s ) + H (A ,s ), (4) 

and for t = 1, . . . , T, the total supplier cost at stage t is given by 

C(A u s t ) + H(A t _ u A t ,s t ). (5) 

In contrast to existing dynamic models for electricity markets with an exogenous demand process 
(Cho and Meyn 2010a,b), our dynamic game-theoretic model incorporates the consumer reactions 
to price fluctuations, and allows us to study the impact of pricing mechanisms on consumer behavior 
and economic efficiency. Through a dynamic game-theoretic formulation, Kizilkale and Mannor 
(2010) study the tradeoff between economic efficiency and price volatility. Our model is different 
from the one studied in Kizilkale and Mannor (2010) in the following respects: 

1. Our model allows the generation cost to depend on an exogenous state, and therefore can 
incorporate supply-side volatility due to uncertainty in renewable electricity generation. As an 
example, consider a case where the exogenous state, s t , represents the electricity generation from 
renewable resources at stage t. Then the demand for conventional generation is A t — s t . Suppose 
that the system reserve is proportional to the system load, say, SA t for some constant 5 > 0. The 
cost function, C(A t ,R t ,s t ), then depends only on the output of conventional generating units, 
A t — s t , and the system reserve, 5A t . The ancillary cost occurred at stage t depends on the system 
reserve and the outputs of conventional generating units at stages t—\ and t, and is therefore a 
function of A t _x, A t , s t -i, and s t . 

2. More important, instead of penalizing each consumer's attempt to change her own action 
across time, the ancillary cost function in our model penalizes the change in the aggregate demand 
by all consumers. The change in a single consumer's action may harm or benefit the social welfare, 
while the volatility of the aggregate demand is usually undesirable. 
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The main feature of our model is the ancillary cost function H, which makes the supplier cost 
nonseparable over time. In an electric power system, the ancillary cost function models the costs 
associated with the variability of conventional thermal generator output, such as the energy cost 
of peaking plants and the redispatch cost. Note that the ancillary cost is not necessarily zero 
when A t < A t _i, because thermal generating units have ramping-down constraints, and because a 
decrease in renewable electricity production may lead to an increase of the system reserve, even 
if A t < A t _i. The presence of the ancillary cost function makes conventional marginal cost pricing 
inefficient (cf. Example 1 in Section 3). 

To keep the model simple, we do not incorporate any idiosyncratic randomness in consumer 
state evolution. Thus, besides the randomness of consumer types (initial states), the only source 
of stochasticity in the model is the exogenous state s t . 

To effectively highlight the impact of pricing mechanisms on consumer behavior, as well as 
on economic efficiency and demand volatility, we have made the following simplifications and 
assumptions for the power grid: 

(a) As in Cho and Meyn (2010a), we assume that the physical production capacity is large 
enough so that the possible changes of the generation capacity are not constrained. 

(b) Transmission capacity is large enough to avoid any congestion. We also assume that the cost 
of supplying electricity to consumers at different locations is the same. Therefore, a common price 
for all consumers is appropriate. 

(c) We use a simplified form of ancillary cost functions, H(A t _ 1 ,A t ,R t _ 1 ,R t ,s t ), to approxi- 
mate the supplier ancillary cost. In Appendix A, we present a numerical example to justify this 
approximation. 

3. The Pricing Mechanism 

The marginal cost pricing mechanism discussed in Section 1 charges a time-varying unit price on 
each consumer's demand. As demonstrated in the following example, a time-varying price that 
equals the supplier's instantaneous marginal cost may not achieve social optimality in a setting 
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that includes ancillary costs. For this reason, we propose a new pricing mechanism that takes into 
account the ancillary cost associated with a consumer's demand at the previous stage. 

Example 1. Consider a two-stage deterministic model with one consumer and one supplier. At 
stage t, the consumer's utility function is U t : [0, oo) — > [0,oo). Let a t denote the demand at stage 
t, and let a= (a ,ai). Let g t denote the actual generation at stage t, and let g = (<7o>Si)- Two unit 
prices, p and p 1 , are charged on the consumption at stage and 1, respectively. Let p = (po,Pi)- 
The consumer's payoff-maximization problem is 

Maximize U (a ) — Poa + Ui(ai) — piai. (6) 

a 

Let H be identically zero, and let the ancillary cost function at stage 1 depend only on the 
difference between the supply at the two stages. That is, the ancillary cost at stage 1 is of the form 
H{g 1 — go). The supplier's profit-maximization problem is 

Maximize p g Q +pi#i - C(g ) - C(g l ) - H(g 1 - g ). (7) 
g 

The social planner's problem is 

Maximize U (a ) + J7i(ai) - C(g ) - C(g 1 ) - H(g 1 - g ) 

( a .g) (8) 
subject to a = g. 

Now consider a competitive equilibrium, (a, g, p), at which the vector a solves the consumer's 

optimization problem (6), the vector g solves the supplier's optimization problem (7), and the 

market clears, i.e., a = g. Suppose that the utility functions are concave and continuously differen- 

tiable, and that the cost functions C and H are convex and continuously differentiable. We further 

assume that iT'(O) = 0, and that for t = 0,1, UtfO) > C"(0), U' t {B) < C'{B). Then, there exists a 

competitive equilibrium, (a, g,p), which satisfies the following conditions: 

jUo(a a )=p , ( C'(a )-H'(a 1 -a )=Po, 

\U' 1 (a 1 )=p 1 , \c , (a 1 ) + H / (a 1 -a a )=p 1 . 

We conclude that the competitive equilibrium solves the social welfare maximization problem in 
(8), because it satisfies the following (sufficient) optimality conditions: 

U^a ) = C'(a )-H'(a 1 -a ), Ufa) = C '(d) + H' '(fli - a ) , 
ao = 9o, ai= gi- 
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However, we observe that the socially optimal price p does not equal the supplier's instantaneous 
marginal cost at stage 0, C'(a ). Hence, by setting the price equal to C'(a ), as would be done in 
a real-time balancing market, we may not achieve social optimality. More generally, marginal cost 
pricing need not be socially optimal because it does not take into account the externality conferred 
by the action a on the ancillary cost at stage 1, H(a\ —a a ). At a socially optimal competitive 
equilibrium, the consumer should pay 

(C"(a ) - ff'(oi - a ))a + (C^) + H' {a, - a ))a u 

i.e., the price on a should be the sum of the supplier marginal cost at stage 0, C'(a ), and the 
marginal ancillary cost associated with a , — H'(ai — a ), which is determined at the next stage, 
after a± is realized. □ 

Before describing the precise pricing mechanism we propose, we introduce a differentiability 
assumption on the cost functions. 

Assumption 1. For any sES, C(-,s) and H (-,s) are continuously differentiable on [0, oo). For 
any (A 1 , s) € A x S 2 , H(A,A',s) and H(A',A,s) are continuously differentiable in A on [0,oo). 5 

Inspired by Example 1, we introduce prices 

p t = C'(A t ,s t ), t = 0,...,T, (11) 

and 

dH(A t ^A u St) _dH(A t _ 1 ,A t ,s t ) 
Qt- ^ , wt- Mt , t-l,...,T. (12) 

At stage 0, we let q = and w = Hq(A ,s ). Under the proposed pricing mechanism, consumer 
i's payoff at stage t is given by 

U t (x itt ,s t ,a itt ) - (pt + w t )a l ,t-qta l ,t-i- (13) 
5 At the boundary of the domain, 0, we require continuity of the right-derivatives of C, Ho, and H. 
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Note that p t + w t is the supplier marginal cost at stage t (including the marginal ancillary cost). 
The proposed pricing mechanism charges consumer i an additional price q t on her previous demand, 
equal to the marginal ancillary cost with respect to a iit _i. 

We now define some of the notation that we will be using. For t = 1, . . . ,T, let yi, t = (ctj it _i, Xi tt ) 
be the augmented state of consumer i at stage t. At t = 0, let y ii0 = x ifi . For stage t, let y t 
be the set of all possible augmented states. In particular, we have 3^o = X , and y t = A x X t , for 
t = l,...,T. 

Let A n (D) be the set of empirical probability distributions over a given set D that can be 
generated by n samples from D. (Note that empirical distributions are always discrete, even if D 
is a continuous set.) Let f t G A n (y t ) be the empirical distribution of the augmented state of all 
consumers at stage t, and let f_ iit G A n _ 1 (y t ) be the empirical distribution of the augmented state 
of all consumers (excluding consumer i) at stage t. We refer to f t as the population state at 
stage t. Let u t G A n (^4) denote the empirical distribution of all consumers' actions at stage t, and 
let n_ ijt G A n _i(^4) be the empirical distribution of all consumers' (excluding consumer i) actions 
at stage t. 

For a given n, it can be seen from (11) and (12) that the prices, and thus the stage payoff in 
(13), are determined by the current global state, s t , consumer i's current augmented state, y iit , 
and current action, a i;t , as well as the empirical distributions, /_ i|t and u_ iit of other consumers' 
current augmented state and action. Hence, for a certain function tt(-), we can write the stage 
payoff in (13) as 

n(yi,t,St,a itU f-i, t ,u-i >t ) = U t (xi >t ,s t ,a itt ) - (p t +w t )a ht - g t a»,t-i- (14) 
4. A Continuum Model and Dynamic Oblivious Strategies 

To study the aggregate behavior of a large number of consumers, we consider a nonatomic game 
involving a continuum of infinitesimally small consumers, indexed by i G [0,1]. We assume that 
(under state s ) a fraction i] so of the consumers has initial state x. In a nonatomic model, any 
single consumer's action has no influence on the aggregate demand and the prices. We consider a 
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class of strategies (dynamic oblivious strategies) in which a consumer's action depends only on the 
history of past exogenous states, h t = (s , • • • , s t ), and her own current state 6 , i.e., of the form 

ai, t = vt{xi,t,h t ). 

Suppose that consumer i uses a dynamic oblivious strategy V = (u Q , . . . , u T )- Since there is no 
idiosyncratic randomness, given a history h t , the state x iit of consumer i at stage t depends only 
on her initial state x ifl . That is, there is a mapping l v<ht : X — > X t , such that x ut = lv,h t {%i,o)- 
Therefore, we can specify the action taken by a dynamic oblivious strategy in the alternative form 

Oi,t = v t (xi,o, h t ) = v t (lv,h t (x ifi ),h t ). (15) 

We refer to v = (u , . . . , u T ) as a dynamic oblivious strategy, and let 2J be the set of all such 
strategies. 

An alternative formulation involving strategies that depend on consumer expectations on future 
prices would lead to a Rational Expectations Equilibrium (REE), an equilibrium concept based on 
the rational expectations approach pioneered by Muth (1961). In our continuum model, since the 
only source of stochasticity is from the exogenous state s t , future prices under any given strategy 
profile, are completely determined by the history h t . Therefore, it is reasonable to expect that 
strategies of the form (15) will lead to an equilibrium concept that is identical in outcomes with a 
REE (cf. the discussion in Section 4.2). 

Before formally defining a Dynamic Oblivious Equilibrium (DOE), we first provide some of the 
intuition behind the definition. In a continuum model, if all consumers use a common dynamic 
oblivious strategy v, the aggregate demand and the prices at stage t depend only on the history 
of exogenous states, h t = (s ,...,s t ). A dynamic oblivious strategy v is a DOE (cf. the formal 
definition in Section 4.2) if it maximizes every consumer's expected total payoff, under the sequence 

6 Note that a dynamic oblivious strategy depends only on the consumer's current state, instead of her augmented 
state. As we will see in Section 4.2, in a continuum model, since any single consumer has no influence on the prices, 
a best response or equilibrium strategy need not take into account the action taken at the previous stage. 
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of prices that v induces. In Section 4.3, we associate a continuum model with a sequence of n- 
consumer models (n = 1,2,...), and specify the relation between the continuum model and the 
corresponding n-consumer model. 

4.1. The sequence of prices induced by a dynamic oblivious strategy 

Let h t = (s ,...,s t ) denote a history up to stage t, and let H t = S t+1 denote the set of all possible 
such histories. Recall that in a continuum model, given an initial global state s , the distribution 
of consumers' initial states is r] SQ . Therefore, under a history h t , if all consumers use the same 
dynamic oblivious strategy u, then the average demand is 

A\ v ,ht = ^2 Vs (x) ■ v t {x,ht). (16) 

xex 

We now introduce the cost functions in a continuum model. Let C : R x S — > [0, oo) be a primary 
cost function. Let H : M 2 x S 2 — > [0,oo) be an ancillary cost function at stage t > 1, and let H Q : 
R x S — > [0, oo) be an ancillary cost function at the initial stage 0. 

Given the cost functions in a continuum model, we define the sequence of prices induced by a 
dynamic oblivious strategy as follows: 

Pt\u,ht =C'(A t \^ ht ,s t ), qo\v,h = ®> w o\v,h = H' (A ^j H) ,s ), (17) 

and for t > 1 , 

_ 9H ^-A t _i| J/i / lt _ 1 ,i4. t | 1/i / lt ,s t ^ ^ dH ^A t _i\ Viht l ,A t \ Viht ,s^j 

Qt\uM = » Wt\v,ht = 77t • ( 18 ) 

0-^t-\\v,h t -l <J A t\v,ht 

4.2. Equilibrium strategies 

In this subsection we define the concept of a DOE. Suppose that all consumers other than i use a 
dynamic oblivious strategy v. In a continuum model, consumer i's action does not affect the prices. 
If all consumers except i use a dynamic oblivious strategy u, consumer i's oblivious stage value 
(the stage payoff in a continuum model) under a history ht and an action o^t, is 

Tti,t{y%,t,ht,a-i,t I v) = U t (x itt ,s t ,a h t) - {p t \u,ht+^t\u,h t ) a i,t-Qt\u,h t ai,t-i, (19) 
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where the prices, Pt\v,h t , Wt\v,h t i an d Qt\v,h t i are defined in (17) and (18). Since a single consumer's 
action cannot influence q t , the last term in (19) is not affected by the action a ut , and the decision 
a ut at stage t need not take a^ t _i into account, but should take q t+1 into account. 
Consumer z's oblivious stage value under a dynamic oblivious strategy z>, is 7 

ni, t (yi,t,h t | v,v) = TT iit (y iit ,h t ,u t (x ifi ,h t ) \ v). (20) 

In particular, we use Tri,t(yi,t,h t \ v-, v) to denote the oblivious stage value of consumer % at stage t, if 
all consumers use the strategy v. Given an initial global state s and an initial state of consumer i, 
x lfi , her oblivious value function (total future expected payoff function in a continuum model) 
is 

V ifi (xi, ,s | u,u) = E i^2jr iiT (y^ T ,h T | z>,zv) j , (21) 
where the expectation is over the future global states, {s T }^ =1 . 
Definition 1. A strategy v is a Dynamic Oblivious Equilibrium (DOE) if 

sup Vi i0 (xi i0 ,s | 0,is) = Vi.o(x ifi ,s | v,v), Vx ifl £X Q , Vs eS. 

A DOE is guaranteed to exist, under suitable assumptions, and this is known to be the case for 
our model (under our assumptions), and even for a more general model that includes idiosyncratic 
randomness (Bergin and Bernhardt 1992). The DOE, as defined above, is essentially the same 
concept as the "dynamic competitive equilibrium" studied in Bodoh-Creed (2010), which is defined 
as the nonatomic equivalent of an MPE, in a continuum model. At a DOE, the beliefs of all 
consumers on future prices are consistent with the equilibrium outcomes. Therefore, a DOE is 
identical in outcomes with a Rational Expectations Equilibrium (REE). 

In future electricity markets, consumers may form rational expectations of future prices through 
an adaptive learning process, or they may receive price estimates from utilities and/or the inde- 
pendent system operator through advanced metering infrastructures. (In Appendix B, we provide 

7 Recall that the initial state (the type) of consumer i, x»,o, is included in its state Xi,t, for any t. 
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some discussion of a possible implementation of the proposed real-time pricing mechanism.) If 
so, a REE (equivalently, a DOE) will be a plausible outcome of such a market. Furthermore, we 
will show (Theorem 2) that under the proposed pricing mechanism, and under certain convexity 
assumptions, a DOE is socially optimal for the continuum model. 

For a dynamic market model with aggregate profit shocks, Weintraub et al. (2010) introduce a 
concept of "extended oblivious equilibrium" at which every firm's strategy depends on its current 
state and on the recent history (as opposed to the full history) of the aggregate shock. The extended 
oblivious equilibrium is computationally tractable; however, an equilibrium strategy may not be 
an approximate best response for every firm, even if the number of firms is large (cf. the error 
bounds derived in Section 8.3 of Weintraub et al. (2010)). 

Note that the definition of a DOE strategy requires optimality (attaining the supremum in 
Definition 2) only along the equilibrium path (Bodoh-Creed 2010). Thus, a DOE is similar in spirit 
to the "self-confirming equilibria" in Fudenberg and Levine (1993) and the "subjective equilibria" 
in Kalai and Lehrer (1995), in which each agent forms correct beliefs about her opponents only 
along the equilibrium path. 

4.3. The n-consumer model associated with a continuum model 

We want the cost functions in a continuum model to approximate the cost functions in an n- 
consumer model. Since the continuum of consumers is described by distributions over [0,1], the 
demand given in (16) can be regarded as the average demand per consumer. To capture this 
correspondence, we assume the following relation between the cost functions in a continuum model 
and their counterparts in a corresponding n-consumer model. 

Assumption 2. For any n£N, any s G5, and any s in S 2 , we have 

C n (A,s) = nC[ -,s ) , H?(A,8)=nH (-,a), H n (A,A',s) = nH(-,—,s 
\n ) \n J \n n 

where the superscript n is used to indicate that these are the cost functions associated with an 
n-consumer model. 
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Assumption 2 implies that 



(C n )'{A,s) = C'(A/n,s 



(H-)'(A,s) = H' (A/n,s), s£S, 



and 



dH n (A,A',s 
OA 



■) _ dH(A/n,A'/n,s) 



d{A/n) 



dH n (A,A',s) _ dH(A/n,A'/n,s 
dA' ~ d{A'/n) 



VseS 2 



i.e., the prices in the continuum model at the average demand equal the prices in the corresponding 
re-consumer model. 

5. Approximation in Large Games 

In this section, we consider a sequence of dynamic games, and show that as the number of consumers 
increases to infinity, a DOE strategy for the corresponding continuum game is asymptotically 
optimal for every consumer (i.e., an approximate best response), if the other consumers follow 
that same strategy. In the rest of the paper, we often use a superscript re to indicate quantities 
associated with an n-consumer model. 

Suppose that all consumers except i use a dynamic oblivious strategy v. Given a history h t and 
an empirical distribution /™ lt , we use v(h t ,f™ it ,v) to denote the empirical distribution, u"L it , of 
the actions taken by consumers excluding i. In an n-consumer model, suppose that consumer i 
uses a history-dependent strategy k" = {n"}J =a of the form 



while the other consumers use a dynamic oblivious strategy v. Let ^ denote the set of all possible 
history-dependent strategies n n for the n-consumer model. Note that since all other consumers use 
an oblivious strategy z/, f^ it is completely determined by u, f" i0 , and h t . 
The stage payoff received by consumer i at time t is 



(22) 



v) = vr" (y iit , s t , a itt , v(h u /™ i>t , v)) 



(23) 
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where a iit = K™(yi,t, h t , f™ i t ), and the stage payoff function on the right-hand side is given in (14). 
Given an initial global state, s , and consumer i's initial state, Xi, , consumer i's expected payoff 
under the strategy n n is 

V^ Q (x ifi ,s I K n ,u) =E j^TT^^A, A t I «",")} , (24) 

where the expectation is over the initial distribution f™ i and over the future global states, {s t }J =1 . 
In particular, we use V™ (x ii0 , s \ v, v) to denote the expected payoff obtained by consumer % if all 
consumers use the strategy v. 

Definition 2. A dynamic oblivious strategy v has the asymptotic Markov equilibrium 

(AME) property (Adlakha et al. 2011), if for any initial global state s € S, any initial consumer 
state Xi.o G %o, and any sequence of history-dependent strategies {k™}, we have 

limsup (V" (x ifi , s I n n , v) - V" (x ifi , s \ u, v)) < 0. 

We will show that every DOE has the AME property, under the following assumption, which 
strengthens Assumption 1. 

Assumption 3. We assume that: 

3.1. The following four families of functions, of A, {C'(A,s) : s G S}, {H' (A,s) : s 6 S}, 
{dH{A,A',s)/dA : (A' ,s) eix S 2 }, and {dH(A' , A,s)/dA : (A',s) eix S 2 }, are uniformly 
equicontinuous on [0,oo). 8 

3.2. The marginal costs are bounded from above, i.e., 

\C'(A,s)\<P, \H' (A,s)\<P, V(A,s)eAxS, 

and 

dH{A,A',s) 
dA 

where P is a positive constant. 

8 A sufficient condition for this assumption to hold is to require a universal bound on the derivatives of the functions 
in each family. 



<P, 



dH(A',A,s) 
dA 



<P, \/(A',s)£AxS 2 
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3.3. The utility functions, {U t (x, s,a)}J =0 , are continuous in a and bounded above, i.e., 
U t (x,s,a)<Q, t = 0,...,T, V(x,s,a)eX t xSxA, 

where Q is a positive constant. 

Combining with Assumption 2, Assumption 3.1 implies that for any e > 0, there exists a 5 > 
such that for any positive integer n, if \A — A\ < nS, then 

\(C n )'(A,s)-(C n )'(A,s)\<e, \(HZ)'(A,8)-(HSy(A,8)\<e, Vs £ S, (25) 

and for any (A, s) £ A x S 2 , 

dH n (A,A',s) dH n (A,A',s) 
dA dA 

Note that the boundness of the cost function derivatives implies the Lipschitz continuity of the 
cost functions. Combining with Assumption 2, for any pair of real numbers (A, A), and any positive 
integer n, we have 

\C n (A,s)-C n (A,s)\ <P\A-A\, \H£(A,s)-HS(A,s)\ <P\A-A\, Vs G S, (27) 

and for any (A, s) <E A x S 2 , 

\H n (A,A',s)-H n (A,A',s)\<P\A-A\, \H n (A , A,s) — H n (A ,A,s) \ <P\A — A\. (28) 

We argue in the following theorem that a DOE strategy approximately maximizes a consumer's 
expected payoff (among all possible history- dependent strategies) in a dynamic game with a large 
but finite number of consumers, if the other consumers also use that strategy. 

Theorem 1. Suppose that Assumptions 2-3 hold. Every DOE has the AME property. 

Theorem 1 is proved in Appendix C. Various approximation properties of nonatomic equilibrium 
concepts in a continuum game have been investigated in previous works. Sufficient conditions 
for a stationary equilibrium (an equilibrium concept for a continuum game without aggregate 



dH n (A',A,s) dH n (A',A,s) 



dA 



8A 



<e. 



(26) 
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uncertainty) to have the AME property are derived in Adlakha et al. (2011). For a continuum 
game with both idiosyncratic and aggregate uncertainties, Bodoh-Creed (2010) shows that as the 
number agents increases to infinity, the actions taken in an MPE can be well approximated by 
some DOE strategy of the nonatomic limit game. Note, however, that in a general n-consumer 
game, even if all consumers take an action that is close to the action taken by a DOE strategy of 
the nonatomic limit game, the population states and the prices in the n-consumer game can still 
be very different from their counterparts in the nonatomic limit game. Therefore, without further 
assumptions on the consumers' state transition kernel (e.g., continuous dependence of consumer 
states on their previous actions), the approximation property of a DOE on the action space does 
not necessarily imply the AME property of the DOE. 

6. Asymptotic Social Optimality 

In Section 6.1, we define the social welfare associated with an n-consumer model and with a con- 
tinuum model. In Section 6.2, we show that for a continuum model, the social welfare is maximized 
(over all symmetric dynamic oblivious strategy profiles) at a DOE, and that for a sequence of 
n-consumer models, if all consumers use the DOE strategy of the corresponding continuum model, 
then the social welfare is asymptotically maximized, as the number of consumers increases to 
infinity. 

6.1. Social welfare 

In an n-consumer model, let x t = (xi it , . . . , x„ tt ) and a t = (ai jt , . . . , a„ jt ) be the vectors of consumer 
states and actions, respectively, at stage t. Under the current global state s t , the social welfare 
realized at stage t is 

n 

W t "(x t ,s t ,a t ) = -C n (A t , 8 t ) - H n (A t _ u A u s t ) + £ U t (x itt , s t , a M ), t = l,...,T, (29) 

i=l 

and at stage 0, the social welfare is 

n 

W t n (x , so, a ) = -C n (A , so) - H r l (A ,s ) + J] U (x il0 , s , a i>0 ). (30) 
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Because of the symmetry of the problem, the social welfare at stage t depends on x t and a t only 
through the empirical distribution of state-action pairs. In particular, under a symmetric history- 
dependent strategy profile K n = (k u , . . . ,K n ) (cf. the definition of a history-dependent strategy in 
Eq. (22)), we can write the social welfare at time t (with a slight abuse of notation) as W™(f™,h t \ 
K n ). Given an initial global state s and an initial population state f$, the expected social welfare 
achieved under a symmetric history-dependent strategy profile K n is given by 

VWoVo I K n ) = W n (r \s | K n ) + E^J2WT(fr,ht |oJ, (31) 

where the expectation is over the future global states {s t }f =1 - In particular, we use Wq (/o,So \ v n ) 
to denote the expected social welfare achieved by the "symmetric dynamic oblivious strategy 
profile", v n = (u, . . . ,u). 

In a continuum model, suppose that all consumers use a common dynamic oblivious strategy v. 
Given an initial global state s , the expected social welfare is 

W ( So | v) = W (s \v)+V^W t {h t | v)\ , (32) 

where the expectation is over the future global states, {s t }J =1 . Here, W t (h t \ v) is the stage social 
welfare under history h t : 

W t (h t | u) = -C{A t \ vM ,s t ) - HiAt^^h^At^h^st) 



+ ^2 r )s {x)U t (l Vlht (x),St,i>t(x,ht)), t = l,...,T, 

x£X 



(33) 



where l v , ht maps a consumer's initial state into her state at stage t, under the history h t and the 
dynamic oblivious strategy v. The social welfare at stage is given by 

W (s | v) = -C(A oWM ,s a ) - H (A oW j H) ,s ) + ^2 Vs (x)U (x,s Q ,u Q (x,s Q )) . (34) 

x£X 

6.2. Asymptotic social optimality of a DOE 

We now define some notation that will be useful in this subsection. Since there is no idiosyncratic 
randomness, given a history h t , the state of consumer i at stage t depends only on her initial state 
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Xj.o, and her actions taken at r = 0, . . . ,t — 1. At stage t > 1, the history /i t and the transition 
function z^t+i = r(x iit , a i;t , s t+ i) define a mapping k ht : X x .A* — > Z: 

%i,t = kh t {Xifl,difi, . . . , aj jt _i), i = l,...,T. (35) 

Given an initial state x^o, consumer i's total utility under a history /i t can be written as a function 
of her actions taken at stages r = 0, . . . , t: 

t 

U ht (x i<0 ,a ifi , a iit ) = U t (x ifi , s , a ifi ) + ^ U t (x ifi , k hr (x ifl , a ifi , a ijT _i), s r , a, ;T ). (36) 

T=l 

Before proving the main result of this section, we introduce a series of assumptions on the 
convexity and differentiability of the cost and the utility functions. 

Assumption 4. We assume the following. 

4.1. Foranys^S, C(-,s) is convex; for any s G S 2 , H(A,A',s) is convex in (A,A r ). 

4.2. For any h T G %t and any Xi_ G ^o? function defined in (36) is concave with respect to 
the vector (o^o, • • • , Oj,t)- 

4.3. For any t>\, any h t G fi t , and any x u0 G X , the function k ht defined in (35) is monotonic 
in a itT , for t = 0, . . . , t — 1; further, its left and right derivatives with respect to a UT exist, for 

T = 0,...,t- 1. 

4.4. For t>l, and for any (x,s,a) G X x S x A, the left and right derivatives of the utility 
function U t (x,z,s, a) in z exist. 

Assumption 4.1 is standard. If the utility function is concave in a, Assumption 4.2 requires that 
the transition function kh t preserves concavity (a linear function would be an example). Note that 
Assumptions 4.1 and 4.2 guarantee that in both models (a dynamic game with a finite number 
of consumers, and the corresponding continuum game), the expected social welfare (consumer 
i's expected payoff) is concave in the vector of actions taken by all consumers (respectively, by 
consumer i). Assumptions 4.3 and 4.4 ensure the existence of left and right derivatives of the 
expected social welfare given in (32), with respect to the actions taken by consumers. An example 
where Assumptions 4.2-4.4 hold is given next. 
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Example 2. Consider appliances such as Plug-in Hybrid Electric Vehicles (PHEVs), dish washers, 
or clothes washers. For such appliances, a customer usually only cares whether a task is completed 
before a certain time. 

Given an initial state (type) of consumer i, x i , let D(x u0 ) and T(x ifi ) indicate her total desired 
demand and the stage by which the task has to be completed, respectively. Under a given history 
h t , the total utility accumulated by consumer i until time t is assumed to be of the form 



for some function Z. If for every x ii0 G X , Z(x ifi , ■ ) is nondecreasing and concave, then Assumption 
4.2 holds. At stage t = 0, we have 

U (x ifi ,So,a it0 ) = Z (x ifi ,m.m{D(x it0 ),a ifi }) . 



Ut(xi t o,Zi >t ,St,ai t t) = Z (x it o, uim{D(x ifl ),a l . t + z l . t }) - Z (x i;0 , min {D(x ifl ),z itt }) . 

For t > T(x l . ) + 1, we let z i)t = D(x ij0 ), and let U t (xi, t ,s t ,a it t) be identically zero. Suppose that 
for every x it0 G X , the right and left derivatives of Z(x ifi , ■ ) exist. Then, Assumptions 4.3 and 4.4 
hold. □ 

Theorem 2. Suppose that Assumptions 2-4 hold. Let v be a DOE of the continuum game. Then, 
the following hold. 

(a) In the continuum game, the social welfare is maximized (over all symmetric dynamic obliv- 
ious strategy profiles) at the DOE, i.e., 9 



where 23 is the set of all dynamic oblivious strategies. 

9 Note that we are only comparing the social welfare under different symmetric dynamic oblivious strategy profiles, 
where all consumers are using the same dynamic oblivious strategy (y or -&). This is no loss of generality because 
under Assumption 4, the social welfare in a continuum game is a concave function of the collection of consumer 
actions taken under the different histories. Hence, it can be shown that the optimal social welfare can be achieved by 
a symmetric dynamic oblivious strategy profile. 




For t = 1, . . .,T(x ifi ), we let z i>t = ^ 



,t-i , 
, T=0 o liT , and 



W (s I v) = sup tfG2 , WqOo I Vs e S, 
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(b) For a sequence of n-consumer games, the symmetric DOE strategy profile, v n = (u, . . . , v), 
approximately maximizes the expected social welfare, as the number of consumers increases to 
infinity. That is, for any initial global state s , and any sequence of symmetric history- dependent 
strategy profiles {«"}, we have 10 



where the expectation is over the initial population state, ffi . 

The proof of Theorem 2 is given in Appendix D. 
7. Conclusion and Future Directions 

In an electric power system, load swings may result in significant ancillary cost to suppliers. Moti- 
vated by the observation that marginal cost pricing may not achieve social optimality in electricity 
markets, we proposed a new dynamic pricing mechanism that takes into account the externality 
conferred by a consumer's action on future ancillary cost. Besides proposing a suitable game- 
theoretic model that incorporates the cost of load fluctuations and a particular pricing mechanism 
for electricity markets, a main contribution of this paper was to show that the proposed pric- 
ing mechanism achieves social optimality in a dynamic nonatomic game, and approximate social 
optimality for the case of finitely many consumers, under certain convexity assumptions. 

To compare the proposed pricing mechanism with marginal cost pricing, we presented a numerical 
example in which the demand increases sharply at the last stage. In this example, the proposed 
pricing mechanism creates a stronger incentive for consumers to shift their peak load than marginal 
cost pricing, through an additional negative price charged on off-peak consumer demand. As a 
result, compared with marginal cost pricing, the proposed pricing mechanism achieves a higher 
social welfare, and at the same time, reduces the peak load, and therefore has the potential to 
reduce the need for long-term investments in peaking plants. 

10 Under Assumption 4, the social welfare in an n-consumer game is a concave function of the collection of consumer 
actions taken under the different histories. Therefore, sup K n eii Wo(fo: s o | K n ) is also the maximum social welfare 
that can be achieved by a (possibly non-symmetric) history-dependent strategy profile. 
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We believe that the constructed dynamic game-theoretic model, the proposed pricing mechanism, 
and more importantly, the insights provided by this work, can be applied to a more general class 
of markets with friction. As an extension and future work, one can potentially develop and use 
variations of our framework to a market of a perishable product / service where demand fluctuations 
incur significant cost to suppliers. Examples include data centers implementing cloud services that 
suffer from the switching costs to toggle a server into and out of a power-saving mode (Lin et al. 
2011), and large organizations such as hospitals that use on-call staff to meet unexpected demand. 
References 

Al-Najjar N. 2004. Aggregation and the law of large numbers in large economies, Games and Economic 
Behavior, 47, 1-35. 

Al-Najjar N. 2008. Large games and the law of large numbers, Games and Economic Behavior, 64, 1-34. 

Adlakha S., R. Johari, G. Y. Wcintraub, A. Goldsmith. 2011. Equilibria of dynamic games with many 
players: existence, approximation, and market structure, Working paper, Stanford Univerisity. http : 
//www. arxiv . org/ abs/ 1011 . 5537 

Barbose G., R. Wiser, A. Phadke, C. Goldman. 2008. Reading the tea leaves: How utilities in the West are 
managing carbon regulatory risk in their resource plan, Lawrence Berkeley National Laboratory, Tech. 
Rep. LBNL-44E, March 2008. 

Bergin J., D. Bernhardt. 1992. Anonymous sequential games with aggregate uncertainty, Journal of Mathe- 
matical Economics, 21(6), 543-562. 

Bodoh-Creed A. 2010. Mean field approximation of large games, Working paper, Cornell University. 

Bodoh-Creed A. 2011. Approximation of large games with applications to uniform price auctions, Working 
paper, Cornell University. http://www.arts.cornell.edu/econ/ab882/LargeGames.pdf 

Borenstein S., M. Jaske, A. Rosenfeld. 2002. Dynamic pricing, advanced metering and demand response in 
electricity markets, http://escholarship.org/uc/item/llw8d6m4 

Chao H. 2010. Price-responsive demand management for a smart grid world, The Electricity Journal, 23(1), 
7-20. 

Cho I., S. P. Meyn. 2010a. Efficiency and marginal cost pricing in dynamic competitive markets with friction, 
Theoretical Economics, 5(2), 215-239. 



28 



Tsitsiklis and Xu: Pricing of Fluctuations in Electricity Markets 



Cho I., S. P. Meyn. 2010b. A dynamic newsboy model for optimal reserve management in electricity markets, 
Working paper, University of Illinois at Urbana-Champaign. https : //netf iles . uiuc . edu/meyn/www/ 
spm_f iles/Papers_pdf /newsboy2010-8-23.pdf 

Doraszclski, U., A. Pakes. 2007. A framework for applied dynamic analysis in 10. Handbook of Industrial 
Organization, Vol. 3. 

Fudenberg, D., J. Tirole. 1991. Game Theory, The MIT Press. 

Fudenberg D., D. K. Levine 1993. Self-confirming equilibrium, Econometrica, 61(3), 523-545. 

Hocffding W. 1963. Probability inequalities for sums of bounded random variables, Journal of the American 
Statistical Association, 58(301), 13-30. 

Hogan. W. 2010. Demand response pricing in organized wholesale markets, IRC Comments, Demand Reponse 
Notice of Proposed Rulemaking, FERC Docket RM10-17-000. http://www.hks.harvard.edu/fs/ 
whogan/Hogan_ IRC_DR_05 1 310. pdf 

ISO New England Inc. 2010. 2009 Annual markets report, http://www.iso-ne.com/markets/mktmonmit/ 
rpts/other/amr09_f inal_051810.pdf 

ISO New England Inc. 2011. Daily Report, www.ferc.gov/market-oversight/mkt-electric/ 
new-england/201 1/02-201 l-elec-isone-dly.pdf 

Kalai E., E. Lehrer 1995. Subjective games and equilibria, Games and Economic Behavior, 8(1), 123-163. 

Kizilkale A. C, S. Mannor (2010), Regulation and efficiency in markets with friction, Proceedings of the 
Forty-Eighth Annual Allerton Conference, Monticello, IL, USA. 

Lin M., A. Wierman, L. Andrew, E. Thereska. 2011. Dynamic right-sizing for power-proportional data 
centers, Proceedings of IEEE INFOCOM, Shanghai, China. 

Lucas R. E., E. C. Prescott. 1971. Investment under uncertianty, Econometrica, 39(5), 659-681. 

Mansur E. 2008. Measuring welfare in restructured electricity markets, The Review of Economics and Statis- 
tics, 90(2), 369-386. 

Mas-Colell, A., X. Vives. 1993. Implementation in economies with a continuum of agents, The Review of 
Economic Studies, 60(3), 613-629. 

Maskin, E., J. Tirole. 1988. A theory of dynamic oligopoly, I and II, Econometrica, 56(3), 549-570. 



Tsitsiklis and Xu: Pricing of Fluctuations in Electricity Markets 

29 

Milchtaich I. 2004. Social optimality and cooperation in nonatomic congestion games, Journal of Economic 
Theory, 141(1), 56-87. 

Muth J. F. 1961. Rational expectations and the theory of price movements, Econometrica, 29(3), 315-335. 

Roughgardcn T., E. Tardos. 2004. Bounding the inefficiency of equilibria in nonatomic congestion games, 
Games and Economic Behavior, 47(2), 389-403. 

Roozbehani M., M. Dahleh, S. Mitter. 2010. On the stability of wholesale electricity markets under real-time 
pricing, Proceedings of the 49th IEEE Conference on Decision and Control, Atlanta, GA, USA. 

Spees K., L. Lave. 2008. Impacts of responsive load in PJM: load shifting and real time pricing, The Energy 
Journal, 29(2), 101-122. 

Stoft S. 2002. Power system economics: designing markets for electricity, IEEE Press. 

US Department of Energy. 2006. Benefits of demand response in electricity markets and recommendations 
for achieving them: a report to the United States Congress pursuant to Section 1252 of the energy 
policy act of 2005. http://eetd.lbl.gov/ea/ems/reports/congress-1252d.pdf 

Weintraub, G. Y., C. L. Benkard, B. Van Roy. 2008. Markov perfect industry dynamics with many firms, 
Econometrica, 76(6), 1375 1411. 

Weintraub G., C. Benkard, B. Van Roy. 2010. Computational Methods for Oblivious Equilibrium, Operations 
Research, 58(4), 1247-1265. 

Acknowledgments 

The authors are grateful to Prof. Michael Caramanis for discussions and several comments on a draft of this 
paper, and to Prof. Ramesh Johari for discussions and pointers to the literature. This research was supported 
in part by the National Science Foundation under grant CMMF0856063 and by a Graduate Fellowship from 
Shell. 



30 



Tsitsiklis and Xu: Pricing of Fluctuations in Electricity Markets 



Electronic Companion 
A. Approximation of the supplier cost 

In this appendix, we show via simulation that at least in some cases, the supplier cost (including 
the cost of ancillary service) can be captured by a simplified cost function of the form in (5). We 
consider a (T + l)-stage dynamic model with two energy resources, a primary energy resource and 
an ancillary energy resource. It is assumed that the forecast demand is met by the primary energy 
resource (e.g., coal- fired or nuclear power generators), and that at stage t = l,...,T, the deviations 
from the forecast demand, {w t }J =1 , are independent random variables uniformly distributed on 
[— u),<J\. At the initial stage 0, we assume that the forecast error is zero, i.e., w = 0. 

At stage t, let b t denote the difference between the actual output of the primary energy resource 
and the forecast demand, and let d t denote the output of the ancillary energy resource (e.g., oil/gas 
combustion turbines). For simplicity, we will assume that the cost of a positive primary energy 
resource (respectively, ancillary energy resource) is b\ (respectively, 10df). 

Let r b be the ramping rate of the primary energy resource, and r d be the ramping rate of the 
ancillary energy resource. At the initial stage 0, we assume that 6 = w = 0, and d = 0. At stage 
t > 1, if w t < 0, then d t = 0, and we assume that b t = 0, that is, the system operator maintains a 
high level of (potential) output in order to be able to deal with a possible unexpected demand surge 
in the future; if w t > 0, we assume that b t = mm{w t , + r b }, where 6 t -i + r b is the maximum 
possible output of the primary energy resource at stage t, and that d t = mm{w t — b t ,d t _i + r d }. 
The total supplier cost (excluding the cost to meet the forecast demand) is 

C = ]T(& t 2 + 10^). (37) 

t=\ 

For notational convenience, we let (■)+ = max{-,0}. We use the following function to approximate 
the supplier cost: 



T 




(38) 



where d t = min i r d , (0,w t - (wt-i) + - r b ) k and b t = (w t -d t ) + . 
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Figure 2 A simulation experiment with T = 24, and 500,000 trajectories for each cj/rt on the horizontal axis. 

The approximation error is defined by \C — C\/C. The average approximation error (vertical axis) is 
the mean value of the approximation errors of the 500, 000 trajectories. 

The function in (38) well approximates the supplier cost in (37), if for an unexpected demand 
surge at stage t, the system load at the previous stage, w t -i, is met by the primary energy resource, 
and load shedding rarely occurs (so that (w t ) + typically equals b t + d t ). Note that in (38), for each 
stage t, the approximated cost depends only on w t -\ and w t . Therefore, the approximated cost in 
(38) can be written as 

T 

C = Y,{{{w t ) + f + H{w t _ u w t )), (39) 
t=i 

where H(w t _ u w t ) = (b 2 t + 10rf* - (K)+) 2 ) + . 

For different values of the parameters, r b , r^, and w, we evaluate the performance of the approx- 
imation via simulation. Fig. 2 depicts some numerical results of a simulation experiment and we 
can make the following observations: 

1. The main source of approximation error is from the following scenario: at stage t — 1, the 
deviation in demand w t _x is nonpositive, w t > r b , and w t+ \ > 2r b . In this scenario, the output of 
the primary energy source at stage t is r&, which is less than w t . When uj/r b < 2, this scenario never 
occurs and we observe from Fig. 2 that the approximation error is close to zero, regardless of the 
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value of r d . 

2. Comparing the black curve with the red curve in Fig. 2, we observe that when u/r b >3 (when 
r b = 0.05 and oj > 0.15), the approximation error for the case where r d = 0.1 is larger than that for 
the case where r d = 0.25. This is because for the case with r d = 0.1, as ui/r b increases from 3 to 6 
(as oj increases from 0.15 to 0.3), the probability of load shedding increases, which deteriorates the 
performance of the approximation. 

3. Finally, and perhaps most importantly, when the ramping rate of the ancillary energy resource 
is high enough to prevent any load shedding, the approximation error is an increasing function of 
the single parameter u/r b (e.g., the blue curve with circle markers for r b = 0.02, r d = 0.1 and the 
red curve for r b = 0.05, r d = 0.25 merge together in Fig. 2); in this case, we observe from Fig. 2 that 
the approximation error is less than 10% for a wide range of parameter values. 

B. Implementation of the proposed pricing mechanism 

To implement a dynamic real-time pricing mechanism, all consumers should be exposed to time- 
varying prices associated with ex ante estimates of generation costs that reflect system operating 
conditions (p. 81 of Borenstein et al. (2002)) so that they can adjust their demand in accordance 
to real-time prices as well as ex ante price estimates. The mechanism proposed in this paper is not 
an exception. The ex ante estimates of real-time prices can be developed by evaluating statistical 
relationships between historical real-time prices and various factors such as load forecast, weather 
predictions, and expected supply/demand balances (Borenstein et al. 2002). 

We now provide a brief discussion of the details of a possible implementation of the proposed 
pricing mechanism: 

Ex ante price estimates. Suppose that the exogenous state s t is realized at the beginning of each 
stage t; for every possible realization of the trajectory (scenario) of future exogenous states 
{s r }r=t+i) consumers receive corresponding price estimates {p r }*tj, {-u) r }*tj, and {g T }*tJ, from 
utilities and/or the independent system operator. The consumers also know or receive the prob- 
abilities of the different trajectories. With the received price estimates (associated with possible 
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trajectories of future exogenous states) and preset utility functions, each consumer's infrastructure 
solves a dynamic programming problem to maximize her expected payoff over the horizon from 
t to t + T. (The state at time r in this dynamic program is comprised of y iiT , and the history 
(s t , s t +i, ■ ■ ■ , s T .) The dimension of this state space grows with the time horizon T (because of the 
exponentially increasing number of histories) . Unfortunately, this is unavoidable for models of this 
type, and might require some further approximations, e.g., in the spirit of Weintraub et al. (2010). 
Ex post prices. At each stage t, after the realization of the system demands A t _i and A t , consumers 
pay ex post prices (p t ,w t ,q t ) that are determined according to Eqs. (11) and (12). 
Equilibrium. In a market with a large number of price-taking consumers, it is possible to make 
ex ante price estimates (contingent on the realized trajectories) that are close to ex post prices. 
If every consumer maximizes her own payoff in response to these pretty accurate price estimates, 
the resulting outcome should be close to that resulting from a Rational Expectations Equilibrium 
(REE) . The results derived in this paper show that the expected social welfare can be approximately 
maximized, under the proposed mechanism. 

We emphasize here that there remain several challenging implementation issues, e.g., the accu- 
racy of future price estimates and the uncertainty of consumer response to ex ante price estimates. 
For example, Roozbehani et al. (2010) show that if consumers act myopically to highly inaccurate 
price estimates, real-time pricing may result in extreme price volatility. However, we note that 
these challenges are generic to almost all kinds of real-time pricing mechanisms. 

C. Proof of Theorem 1 

We consider a sequence of n-consumer models where n — 1 consumers (all except for consumer 
i) use a DOE strategy v. As the number of consumers increases to infinity, the randomness of 
consumer initial states averages out. Thus, in Step 1 we show that the aggregate demand (in an 
n-consumer model) at a history h t is close to nA t \ u j H (defined in Eq. (16)), with high probability. 
As a consequence, we show in Step 2 that as n — > oo, consumer i's expected payoff associated with 
any sequence of actions can be approximated by her oblivious value defined in (21). Since the DOE 
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strategy v maximizes consumer i's oblivious value among all possible strategies, we argue in Step 
3 that as n — > oo, the maximum expected payoff consumer i can obtain is asymptotically no larger 
than the optimal oblivious value. In Step 4, we show that consumer i's optimal oblivious value 
can be approximately achieved if she uses the DOE strategy v. We finally conclude with the AME 
property of the DOE strategy v (the result in Theorem 1). 

In what follows, we will be using the uniform metric over the set of probability distributions on 
the finite set X . Specifically, if / and /' are two distributions on X , we let 

d(f, /') = ||/ - f\L = max - f'(x)\ . (40) 

Step 1: With high probability, the aggregate demand under a history h t is close to nA t \ v . ht . 
Given an initial distribution f" i0 , and if all consumers (excluding i) use a dynamic oblivious 
strategy u, we write their aggregate demand at a history h t as 

A n _. t = (n-l)J2 f-iA x M*,h t ). 

x£X 

Recall that (cf. (16)) 

A\ v ,h t = ^2 Vs (x)-Vt(x,h t ). 

x£X 

We observe that if d(f" i ,r/ so ) < S/(XB), then at any history h t we have 

A n _ itt -(n-l)A tl „ M \<S(n-l), (41) 

with probability at least 1 — 0(e~ n ). More precisely, since the consumers' initial states are inde- 
pendently drawn according to r/ so , Hoeffding's inequality (Hoeffding (1963)) yields, 

F(d(F?-\r] S0 ) >5/{XB)) <2Xexp{-2(n-l)5 2 /(X 2 B 2 )} , Vs G S, V5 > 0, Vn€N+, (42) 

where X is the cardinality of the set X and ^ s " _1 is an X-dimensional random vector denoting 
the distribution of the initial states of the n—1 consumers (excluding i). 

Step 2: Under a given history hr, consumer i 's expected payoff can be approximated by a cor- 
responding oblivious value, defined in (45). 
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In an n-consumer model, suppose that all consumers other than i use a dynamic oblivious 
strategy v. Given a complete history h T = (s , ■ ■ ■ , s T ), and consumer i's initial state x it0 , we define 
her expected payoff under a history-dependent strategy K n by 

Vftfoo, h T | K n ,u) = E {V^(x ifi , hr, fl ifl | «",!/)} , 

where the expectation is over the initial distribution, f1 i0 , and V^ n (x ii0 , ^t, o I i/) is consumer 
i's payoff under the given initial distribution /2 i , 

T 
4=0 

and where the stage payoff function, vr™ 4 (-), has been defined in (23). Note that given /™ i , and since 
all consumers other than i use a dynamic oblivious strategy, the distribution of their augmented 
states, /™ i t , is completely determined by the history h t . Therefore, given f " i , consumer i's history- 
dependent strategy n n is equivalent to a dynamic oblivious strategy: the action it takes at stage t 
depends only on x ii0 and h t . We can therefore define an oblivious strategy u n (K n , f " i ) such that 

^(« n . f-i,o)( x i,o, ht) = K?(Vi,t, h t , f-i, t ), 

where f" i t is the distribution of the n — 1 consumers' augmented states under the history h t , 
induced from the initial distribution /" i0 by the symmetric oblivious strategy profile (v, ...,v), 
and yi tt is consumer i's augmented state under the history h t , induced from her initial state x ifi 
by the strategy n n . 

In the corresponding continuum model, suppose that all consumers other than i use a dynamic 
oblivious strategy v. For a given complete history hr, we define consumer i's oblivious value under 
an initial distribution , her initial state x ii0 , and the history-dependent strategy n n : 

T 

Vifi(xifi,h T , /™ i;0 | K n ,u) = J2*iAyi,t, ht I v(K n ,F.i,o),v), (44) 

t=o 

where the oblivious stage value function 7r, it (-) is given in (20). We define the expected oblivious 
value for consumer i under the history-dependent strategy tx , as 

11 This is actually the oblivious value achieved by a mixed strategy under the complete history hr - In the continuum 
model, under a history ht, the mixed strategy takes an action ut(K n , f-ifi){xi,o, ht), if the distribution of the n — 1 
consumers' (excluding i's) initial states in the corresponding n-consumer model is realized as f-i.o- 
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V ifi (x ifi , h T | n n ,v) = E [v ifi (x ifi , hr, f n _ lS) | n n ,u)} , (45) 

where the expectation is over the initial distribution, /™ i0 . For any e > 0, in this step we aim 
to show that there exists a positive integer ./V such that for any sequence of history-dependent 
strategies {k™}, 

V it0 (x ifi ,h T \K n ,u)-V^(x ifi ,h T \K n ,v)\<e, Vn>iV, V/i T G^ T , Vx^ £ X . (46) 

For a given s , let S r "~ 1 (^) be the set of /™ i such that d(/2 i)0 )f?s ) ^ ^- To verify (46), we first 
argue that for any e > 0, there exists an positive integer N 1 and some 8 > such that for any 
f-i,o e ^- 1 (<5/(XS)) and any n > iV 1; 

^,o(si,o,/irJ!: i ,ol« n ^)-V^ (si,o,/iT,/!! j ,ol« n ,»')|<e/2, V/j t G^ t , Vx <l0 €^o. (47) 

Under the uniform equicontinuity assumption for the derivatives of the cost functions (see Eqs. 
(25) and (26)), we know that a small deviation of the aggregate demand from A t \ u j H will result in 
prices that are only slightly different from the prices in the continuum model. We also note that 
consumer i cannot take an action larger than B, and her payoff is influenced by other consumers 
only through the prices. For any e > 0, we can find some 8 > and a positive integer N 1 such that 
for any given (x ifi ,h t ), if /™ i G ^"~ 1 (8/(XB)), then the inequality in (41) holds for any history 
h T , which implies that 

|^, t (y M A,<(y M A,/^ Vn>Aq, Vht, (48) 

i.e., consumer i's stage payoff (under the action K"(y ijt , h t , fH l t )) in the n-consumer model is close 
to her oblivious stage value (under the same action «"(yi,t, h t ,f ' n : i t )) in the continuum model, if the 
initial distribution in the n-consumer model, /" i , is close to its expectation. The result in (47) 
follows from Eq. (48) and the definitions in (43) and (44). Note that Q + 2BP is an upper bound 
on the stage payoff that consumer i could obtain, and —2BP is a lower bound on consumer i's 
stage payoff, under Assumption 3. The desired result in (46) follows from (47), and the fact that 
the probability that /" i ^ ^~ 1 (8/(XB)) decays exponentially with n (cf. Eq. (42)). 
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Step 3: The maximum expected payoff consumer i can obtain is asymptotically no larger than 
the optimal oblivious value. 

In this step, we consider the case where all consumers in an n-consumer model except for i use 
a DOE strategy v, and argue that for any sequence of history-dependent strategies {k™}, 

limsup (V" (x ifi ,s | K n ,u) - V ifi (x ifi ,s \ v,v)\ < 0, Vs G5, Vx; ;0 G X , (49) 

where consumer Vs expected payoff, V™ (x, s \ K n , u), is given in (24), and V ifi {x,s \ v,v) is the 
oblivious value function in (21). We first observe that 

V^(x l , ,s \ K n ,v) = nh T \s )-V^(x ifi ,h T \K n ,v), (50) 

where %t(«o) is the set of complete histories commencing at state s , and F(h T \ s ) is the proba- 
bility that the history h T is realized, conditional on the initial global state being s . We define 

V it o(x it0 ,s 1 K n ,v) = ^2 F(h T \s )-Vi fi (x ifi ,h T \K n ,u). (51) 

/i T G« T (s ) 

Note that if n n happens to be a dynamic oblivious strategy, this definition is consistent with the 
definition of oblivious value function in (21). 

For any e > 0, let N be the integer defined in Eq. (46); for any sequence of history-dependent 
strategies {k™}, we argue that 

Vi t o(x it0 ,s | u,u) > V* ¥>(h T \s )-V ifi (x ifi ,h T \K n ,v) 

— 'h T £U T (s ) 

^ E„ ^ , , I s «) • WoKo, hr\K n ,v)- e) (52) 
= V r " (i i ,o,s |K»-£ ) Vre > N, Vx hQ £X Q . 

The DOE strategy is, by definition, maximizes consumer i's oblivious value function among all pos- 
sible dynamic oblivious strategies. The first inequality in (52) follows from the fact that V^ i0 (^i,o, «o I 
K n , v) is a weighted sum of the oblivious values achieved by a family of dynamic oblivious strate- 
gies 12 . The second inequality in (52) is due to (46), and the last equality in (52) follows from (50). 
The desired result, (49), follows. 

12 Note that for a given /™j oi the action taken by k" depends only on Xifi and ht, and that Vifl (a;;,o,/iT | is 
the oblivious value achieved by a mixed strategy; cf. the footnote associated with (45). 
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Step 4: Consumer i's optimal oblivious value can be asymptotically achieved at an n-consumer 
game under a DOE strategy. 

In this step, we consider the case where all consumers in an n-consumer model use a DOE 
strategy i/, and show that 

lim ( Vifi(xifl,s | v) - V" (xifl,s , \ v, v) ) = 0, Vs G Vxj, G X . (53) 

According to (46), with K n = u, for any e > 0, we can find some N such that 

Vifl(x ifi ,h T \v,v)-VP; {xi fi ,h T \v,v) <e, Vn>N, \/h T eH T , Vx 4 , G X Q . 

The desired result in (53) follows from (50) and (51). Theorem 1 follows from (49) and (53). 
D. Proof of Theorem 2 
D.l. Proof of Part (a) 

We will show that in a continuum model, a DOE strategy maximizes the expected social welfare 
among all possible symmetric dynamic oblivious strategy profiles (part (a) of the theorem), i.e., 
that if v is DOE, then 

Wo(s \v) =sup^ eOT Wo(s 1 0), VsoGS. (54) 

Let S and X be the cardinality of S and X , respectively. Given the initial global state s , the 
number of possible histories of length t + 1 is 5*. Hence, the number of all possible histories 
commencing at state s is X)^=o^"- Given an initial global state s , the expected social welfare 
defined in (32) is a deterministic function of the following (X Y^t=a -S^-dimensional action vector: 

(55) 

where H(s ) is the set of positive probability histories commencing at state s . Under Assumption 
4, the expected social welfare defined in (32) is a concave function of the vector in (55). Therefore, 
the following conditions are necessary and sufficient for the action vector (in the form of (55)) 
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associated with the DOE strategy v to maximize the expected social welfare, among all possible 
dynamic oblivious strategies 13 : 



' d + U t (l Vtht (x it0 ),s t ,u t (x it0 ,h t )) , 

-7j , < Pt| Vlfct + U^h, + l r<T • 5 + ^ (X i>0 ), 

if v t (xifi,h t ) < B, 

d-U t (l Vtht (x ifi ),s t ,v t (x ifi ,h t )) _ . 
d _ Vt{xiM >Pt\,M+^M + W ■JwKo)- 

if v t (x ifi ,ht) > 0, 



(56) 



where l v ,h t (%i,o) is consumer i's state, x i)t , under a (positive probability) history /i t and the strategy 
(cf. p. 15), the prices, Pt\ v ,h t and Wt\v,h t are given in (17) and (18), and where, if k hr (-) (cf. the 
definition in (35)) is nondecreasing in a ijt for any t < r < T, then ht (aJi,o) is given by 14 



+ / \ J ~ 9 + U T (x i , z iT , s T , a iT ) d + z i T I 

L T=t+1 + *' T + J 

where the price, Qi+i|i/,/» t+1 , is defined in (18), the expectation is over the future global states, 
{s T }f +1 , Zi. r = k hT (xi, , a»,0) • • • ,«i,r-i) for r > i, and a ijT = u T (x iiT ,h T ) for t > t. The expression (57) 
is the part of the right derivative of the expected social welfare (32) with respect to the action a^ t , 
which reflects the influence of consumer i's action at stage t on the ancillary cost H(A t , A t+1 ,s t+1 ) 
at the next stage, and on her future utility (due to the influence of the action a i>t on the future 
state Zi >T , through the functions k hT (-)). In (56), g~^ v ht (x it0 ) can be defined by replacing the right 
(left) partial derivatives in (57) with left (respectively, right) partial derivatives. 

Given an initial global state s , and the initial state of consumer i, x i)0 , her oblivious value, 
defined in (21), is a deterministic, concave function of the vector 

{vt(x ifl ,h t )} hten{so) (58) 

of actions that she would take at any given stage and for any given history. Since the DOE strategy 
v maximizes consumer i's oblivious value, it is easily checked that the vector in (58) must satisfy 
the conditions (56). Since this is true for any x ii0 G X , we conclude that the action vector (55) 

13 We use the notations d+f and d-f to denote the right and left, respectively, derivatives of a function /. 

14 If for some r > t, kh T (-) is decreasing in a, : t, then the right partial derivative of U T {x i>T ,s T ,a i>T ) with respect to 
Zi ;T in (57) should be replaced by its left partial derivative. 
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(which is comprised by putting together the vectors in (58), for different types in the set X Q ) 
satisfies the conditions (56). Thus, the DOE v satisfies the sufficient condition for optimality and 
the result (54) follows. 

D.2. Proof of Part (b) 

For a given initial global state s , let us fix some initial distribution /q with d(fQ,rj S0 ) < 5, where 
5 is small. In Step 1, we compare the social welfare achieved by various strategy profiles and show 
that 

^W "(/o\ So I «") < l^(s a | n 'ft) « W (so I <T' /o "). 

Here, K n is a general history-dependent strategy profile for the n-consumer model (cf. (22)). The 
symmetric strategy profile # n '^° = [d n ' f ° , . . . , fi n >fo ) is one that maximizes expected social welfare 
given the initial population state ffi. In Step 1, we will argue that can be identified with 

a dynamic oblivious strategy. In the approximate equality we are comparing the expected (over 
future global states, {s t }J =1 ) social welfare under the same oblivious strategy ■d n ^o (hence the same 
sequence of actions for each consumer type x £ X ) under two different initial population states 
(initial distributions of consumer types), /q and rj so . 
Since v is a DOE, part (a) of the theorem implies that 

Wo(so|?T' /l ')<Wo(sok). 

Note that as the number of consumer grows large, with high probability the initial population state 
/o is close to its expectation, r\ Hy In Step 2, we complete the proof of part (b) by showing that 

W (s \u)n^E{WZ(fZ,s \v n )}, 

where the expectation is over the initial population state /q . 

Step 1: If the initial population state is close to its expectation, the optimal social welfare in 
an n-consumer model can be approximated by the social welfare achieved by a dynamic oblivious 
strategy in the corresponding continuum model. 
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In this step, we aim to show that in an n-consumer model, for any given initial global state s 
and any e > 0, there exists some S > such that for any initial distribution ftf with c2(/q ,r] so ) < 5, 
we can find a dynamic oblivious strategy d n <f<8 that satisfies 

W n (/ ",So I K n ) < nW (s | ) + £n , (59) 

for all symmetric history-dependent strategy profiles, n n = (K n ,...,K n ). Given an initial global 
state s and an initial population state f£, we observe that the social welfare, W l (/ ^,s I 
is a deterministic, concave function of the following vector of consumers' actions under different 
histories, 

{< K^(*i.o)AJ^,t)}^ 6W(ao)i x . oeXo , i=1 ,..., nJ (60) 
where m itK n ht :X —^y t maps consumer i's initial state into her augmented state at the history h t , 
under the strategy profile n n , and f™ i t is the distribution of other consumers' augmented states at 
the history h t , under the strategy profile n n . Note that given the initial population state f£, the 
strategy profile n n , and a history h t , the augmented state of consumer i at stage t depends only 
on her initial state x lfi . 

Since the social welfare Wo(/^,So I K ") is concave in the action vector in (60), there exists a 
symmetric solution, tf n >fo = }J =0 , such that if at any history h t £ ~H(s ), all consumers with 

the same initial state take the same action according to 

a i>t = -&t J ° (xi,o,h t ), i = l,...,n, (61) 

then the expected social welfare, W l (/ ^,s | (t n ), is maximized among all possible symmetric 
history-dependent strategy profiles 15 . In (61) we have defined a dynamic oblivious strategy d n >fo 
that maximizes the expected social welfare in the n-consumer model, conditional on the initial 
global state being s , and the initial population state being f£. That is, in an n-consumer model, 
for any given s and f£, there exists a dynamic oblivious strategy d n ' f ° such that 

sup K n W UVo I « = Wo (/o"^o I #"' /o "), (62) 

15 The fact that the suprcmum is attained is a consequence of our continuity assumption and the fact that the various 
variables of interest can be restricted to be in a compact set. 
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where = {-d nJ o , . . . , & n is the corresponding symmetric dynamic oblivious strategy profile. 
To verify (59), it suffices to show that for any e > 0, there exists some 5 > such that for any /q 
with d(f£,r] so ) < 5, 



VW ", s | n '# ) - nW (s | 



< en, 



(63) 



i.e., if all consumers use the strategy i? n, ^o , the difference between the optimal social welfare 
achieved in an n-consumer model and the social welfare achieved in the corresponding continuum 
model can be made arbitrarily small, if the initial population state is close enough to its expectation, 
r) SQ . We next argue that the result in (63) holds for any dynamic oblivious strategy *&. 

To prove (63), we first upper bound the difference between the supplier cost in an n-consumer 
model and that in the corresponding continuum model. Since all cost functions are Lipschitz 
continuous (see Eqs. (27) and (28)), for any e > 0, there exists some 5i > such that if 



A n t -nA tWM ^Xd^n, t = 0,...,T, Vh t £H{s ), 



(64) 



then 



C n (A?,s t )-C n (nA tl # M ,s t ) <ne/(3T + 3), t = 0,...,T, V7i t e H t (so), (65) 



H2(AZ,s Q ) - HZ(nA ol # M ,s ) < ne/(3T + 3), 



(66) 



and for t = 1 



? ■ ■ * ? s 



^(A^A^St) - ^(nA^^nA^^St) <ne/(3T + 3), Vh t eH t (s ), 



(67) 



where H t (so) is the set of all histories of length t + 1 commencing at state s - Given an initial 
population state /q , if all consumers use the strategy the aggregate demand under a history /i t 
is 

xex 

From (16) we observe that if d(/^,?? so ) < 5i, the condition in (64) holds, and then Eqs. (65)-(67) 
are verified. 
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We now show that if the initial population state is close to its expectation, the total utility 
obtained by all consumers is close to its counterpart in the corresponding continuum model. Given 
an initial population state ftf, we write the total utility obtained by all consumers under a history 
h t as 

n 

^2U t (x itt ,s t ,a itt )=n ^2 fo( x ) u t (hj lt {x),$ t (x,h t ),s t ) . 
On the other hand, the utility achieved in the corresponding continuum model is given by 

U t \#,h t = ^2 r 1s {x)U t (U M {x),'& t {x,h t ),s t ) . 

x£X 

We have that if d(/ ",r? so ) < e/(3XQ(T+ 1)), then 

^2^U t {x ijU s u a^)-nU tWM <ne/(3T + 3), t = 0,...,T, \th t tU t (s ), Vn G N + , (68) 

Let 5 = m\n{8 u e/(3XQ(T + l))}. If d(f^\r] S() )<5, from (65)-(68) we have 

W t (h t \ti)-W t n (f?,h t \# n )\<ne/(T + 1), t = 0,...,T, Vh t eU t {s ). 

Eq. (63) follows from the definition of expected social welfare in an n-consumer model (31), and 
in a continuum model (32). The desired result in (59) follows. 

Step 2: Asymptotic social optimality of a DOE. 

In this step, we complete the proof of part (b) of the theorem, using the fact that as the number of 
consumers grows large, with high probability the initial population state is close to its expectation. 
Note that the action space is [0,5], so that \A t \ < nB. Using Assumption 3, C n {-)/n is therefore 
bounded. A similar argument holds for H'^{-)/n and H n (-)/n. Furthermore, the total utility per 
consumer is also bounded. Thus, there exists some constant D that upper bounds |Wo/n|. We 
define $™ Q (5) as the set of initial population states such that d(fQ,r) S0 ) < 5. By the law of large 
numbers, for any pair of positive real numbers, e and 5, we can find an integer ./V such that 

P(F: o = fZy s up\WUfZ,s \K n )\<DF(d(fZ,r ]so )>S)<en, Vn > N. (69) 
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For any e > 0, let S be the positive real number denned in (59), and let iV be the positive integer 
given in (69); for any n> N and any symmetric history-dependent strategy profile n n , we have 

EW(/;, S ok")} 

^ E /o n £ ,„ (a) HK = fo ) ■ HWoVo I *") + ^ 
^ Eyl^) P (^. = # ) • ("W (*o | ^./o" ) + en) + en 
* E/^n W P ( F "o = tf) • (nm(s I +en 
^ E/n^J w P ( F "o = /o l ) • (W "(/ n , ao I ^) + 2en) + en 
<E{WZ(f},s | !/")} + Aen, 
where the first inequality follows from (69), the second inequality is due to (59), the third inequality 

follows from the optimality property of the DOE v (part (a) of the theorem), the fourth inequality 
follows similar to (63) (the proof of Eq. (63) remains valid for any dynamic oblivious strategy), 
and the last inequality follows from (69). 
E. Numerical Results 

In this section we give a numerical example to compare the proposed pricing mechanism with 
marginal cost pricing. The comparison is carried out in terms of DOEs and the resulting social 
welfare under the corresponding continuum model. Towards this purpose, we first define the DOE 
for a continuum model under the marginal cost pricing mechanism, in Section E.l. In Section 
E.2, we consider a two-stage dynamic model in which the consumers' marginal utility and demand 
increase at the second stage. We calculate the equilibria resulting from the two pricing mechanisms, 
and compare the potential of the two pricing mechanisms to improve social welfare and reduce 
peak load. 

E.l. Equilibrium under Marginal Cost Pricing 

In an n-consumer model, at stage t>l, the supplier's marginal cost is 

(Cn\A?,s t ) + dHn{A ^ A7, ° t) =P? + <, t = l,...,T. (70) 
At stage 0, the supplier's marginal cost is 

(C")'K, s ) + (HZ)' (A$, s ) =p n o + <• (71) 
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Under marginal cost pricing, each consumer's stage payoff is 

7r(i/i, t , I t , Oi, t , t£ iit ) = 17 (x ilt , s t , Oi,t) - (p? + O ■ (72) 

where the stage marginal cost, p" + w", is given in (70) and (71), and y iit = (x ijt , a iit _i). 

For marginal cost pricing, we now define the nonatomic equilibrium concept in the corresponding 
continuum model. Suppose that all consumers other than i use a dynamic oblivious strategy v. 
Consumer i's oblivious stage value under marginal cost pricing is given by 

Tfi,t(yi,t,s t ,ft\v,h t i a i,t I v ) = U t (xi tt ,s t ,ai,t) - {pt\u,i H +w t \ v ,h t ) ■ a>i,t, (73) 

where Pt\ v ,h t an d w t \ v _ ht are defined in (17) and (18). Replacing the oblivious stage value function 
in (19) with that given in (73), we can define an equilibrium concept for the marginal cost pricing 
mechanism in a similar way as for the DOE in Section 4. 

E.2. Numerical Example 

In current wholesale electricity markets, we observe that the highest daily wholesale price usually 
occurs when the system load increases quickly (cf. Fig. 1 in Section 1). Inspired by the above 
observation, we construct a two-stage dynamic model, in which the aggregate demand increases 
quickly at the second stage, to compare the performance of the proposed mechanism with marginal 
cost pricing. For simplicity, we assume that there is a continuum of identical consumers indexed by 
i € [0, 1]. Each consumer would like to consume 1 + x and 1.2 — x at the two stages, where x € [0, E]. 
Here, E G [0,0.1] (a given constant) is the amount of electricity demand that can be shifted from 
the second stage to the first stage. The value of E will be called demand substitutability 16 . 

Formally, consumer i's state at each stage denotes the maximum amount of electricity she could 
use at the stage 17 . For a given consumer i, we have x ifi = 1 + E, and her state at stage 1 is 
determined as follows: 

16 There are two types of elasticity of consumers' demand: (i) consumers may curtail their demand at a high price, 
and (ii) they may shift their demand to a less expensive time. The first type of demand response is a price elasticity, 
and the second type is an elasticity of substitution across time. The first type of elasticity is incorporated in our 
model through the utility functions, and the second type of elasticity is incorporated through E. 

17 Since all consumers are of the same type, the consumer state space in this example is a subset of [0,oo). 
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1- if £Ji,o < 1) the maximum amount of electricity she could use at stage 1 is 1.2 — i? + E, i.e., 
x it i = 1.2; 

2. if 1 < a ii0 < Xifi, the maximum amount of electricity she could use at stage 1 is x iS =1.2 — 
E + (x ifi - a it0 ) = 2.2 - a h0 ; 

3. if x^o < ciifi, the maximum amount of electricity she could use at stage 1 is a^i =1.2 — E. 
To summarize, we have 



where the slopes are d = 10 and di = 12. Here, we assumed that the consumers place a larger 
value on electricity during peak hours, and that shifting peak load to off-peak hours hurts con- 
sumer utility. For example, rescheduling kitchen and laundry activities may cause inconvenience 
for residential consumers; similarly, industrial consumers may face higher labor cost premiums for 
off-peak production. 

The primary cost function (cf. Section 2) is C(A, s) = A 2 , for any s. We assume that the capacity 
available at each stage is proportional to the system load, i.e., 



and that the ancillary cost depends only on the difference between the capacity available at two 
consecutive stages. At the second stage (peak hour), we assume that the system operator maintains 
a reserve margin of 10%, i.e., b\ = 1.1. We will consider two different system operator policies: (i) 
the system operator does not forecast the load jump at the second stage, and uses a conservative 
policy under which b = 1.12, and (ii) the system operator predicts the load jump at the second 
stage, and ramps up the system capacity in advance, by letting b Q = 1.2. 



x it i = 1.2 — E + max{0, x ii0 — max{a i;0 , 1}}. 



For each stage t, the utility functions are given by 




if < a ijt < Xi 
if a iit >x itt , 



G t = b t A t , f = 0,l 
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For simplicity, we use a quadratic function to approximate the ancillary cost associated with 
load fluctuations: 

H (A ,s ) = 10(max{& A> - 1.12, 0}) 2 , H(A ,A 1 ,s 1 ) = 20(max{Mi - &oA),0}) 2 , 

where 1.12 represents the capacity available at the stage before the initial stage 18 . We assumed a 
higher coefficient, 20, for the ancillary cost at the second stage, due to the increase of the system 
load. 

For different levels of demand substitutability E, and two different system operator policies (b 
equal to 1.12 or 6 = 1.2), we compare the social welfare (in Section E.2.1) and the peak load (in 
Section E.2.2) resulting from the equilibria of the two pricing mechanisms. 

E.2.1. Social welfare gain. For various levels of demand substitutability (E € [0,0.1]), and 
the two different system operator policies, we calculate the equilibria resulting from the two pricing 
mechanisms. Fig. 3 compares the social welfare achieved by the proposed mechanism and the 
marginal cost pricing mechanism. We observe from Fig. 3 the following. 

1. System operator's policy: When the consumers have a low level of demand substitutability, 
the policy with b = 1.2 achieves a much higher social welfare than the conservative policy (6 = 
1.12), under both the proposed and the marginal cost pricing mechanisms. (This is to be expected, 
because when b = 1.12, and with the demand at stage 1 more or less fixed, the difference b i A 1 — b A 
is necessarily large.) For consumers with a high level of demand substitutability, the policy with 
b = 1.2 achieves a slightly smaller social welfare than the conservative policy (b = 1.12), because 
the policy with b = 1.2 results in a lower price at the second stage than the conservative one, and 
therefore does not provide enough encouragement to the consumers to shift their peak load (cf. 
the discussion in Section E.2.2). 

2. Social welfare gain at a low level of demand substitutability: At a low level of 

demand substitutability, e.g., when E < 0.02, and under the system operator's conservative policy 

18 Suppose that the load at stage "—1" is 1, and that the capacity available at stage — 1 is 1.12, under an average 
reserve margin of 12%. 
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Figure 3 The social welfare achieved by the proposed pricing mechanism, the marginal cost pricing mechanism 
and the flat rate pricing mechanism, as a function of the demand substitutability, E. 

(bo = 1.12), we observe that the proposed pricing mechanism achieves significantly more social 
welfare gain (the social welfare achieved by flat rate pricing 19 is used a reference) than marginal 
cost pricing; if the system operator ramps up the capacity in advance (bo = 1.2), both pricing 
mechanisms achieve approximately the same social welfare as flat rate pricing. 

3. Social welfare gain at a high level of demand substitutability: If the consumers have 
a high demand substitutability, e.g., when E > 0.08, the proposed pricing mechanism achieves 
approximately 5% more social welfare gain than marginal cost pricing under the system operator's 
conservative policy (bo = 1.12); if the system operator ramps up the capacity in advance (bo = 
1.2), the proposed pricing mechanism achieves approximately 50% more social welfare gain than 
marginal cost pricing. 

Let us now derive some insights by considering the special case of zero demand substitutability 

(E = 0) and b = 1.12. The one-stage aggregate demand and the social welfare resulting from the 

19 Under flat rate pricing, consumers pay a fixed (time-invariant) retail price for the electricity they consume. Since 
the average retail price is less than the consumers' marginal utility (see Tables 2 and 4), the payoff-maximizing 
consumer demand at the two stages is 1 and 1.2, respectively. Since all consumers are identical, the aggregate demand 
at the two stages is 1 and 1.2. 
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Table 1 Demand and social welfare (per consumer) at E — and bo = 1.12 



a cti Social welfare 



Flat rate 1 1.2 21.16 

Marginal cost 1 1.2 21.16 
Proposed 1.0901 1.2 21.4735 



Table 2 Price fluctuation at _E = and bo = 1.12. The price p t + w t equals the marginal cost at stage t. 



three pricing mechanisms are given in Table 1. The prices faced by consumers are given in Table 
2, where the average retail price is the ratio of the total money a consumer pays at an equilibrium 
to her total demand during the two stages. 20 Note that under the proposed pricing mechanism, a 
consumer pays 



while she would pay (p + w )a u0 + (p 1 +w 1 )a iil under marginal cost pricing. In general, the price 
qi will be negative and will be even smaller if we were to increase the aggregate demand at the 
second stage. That is, a higher peak load results in a lower price at the first stage, which encourages 
consumers to increase their demand at the off-peak hour, even if they do not derive any additional 
utility from such an increase. In fact, from Table 2 we observe that at the DOE, the proposed pricing 
mechanism offers each consumer a zero total price on a . This may appear illogical at first sight. 
The reason is that due to the conservative reserve policy, with b = 1.12, a demand of o = 1 results 
in a large increase from b a Q to b\a\ and hence a large ancillary cost. The increase of the demand 
a beyond 1 does not provide any utility to the consumer, but reduces the ancillary cost. Thus, 
the counterintuitive choice of a = 1.0901 serves to mitigate a conservative and somewhat deficient 
reserve policy. This suggests that further research is needed that will include an intertemporal 
optimization of the reserve policy as well. 

20 Note that only consumers under flat rate pricing pay this price. We list the average prices for the two real-time 
pricing mechanisms to compare the consumers' expense under different pricing mechanisms. 



Po + w pi + Wi qi Average (retail) price 



Flat rate 
Marginal cost 
Proposed 



2 11.2 - 7.0182 
2 11.2 - 7.0182 
4.4401 6.7609 -4.4401 5.6562 



(p + w + qi)a t .o + {pi +^i)a M , 
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Table 3 Demand and social welfare (per consumer) at E = 0.08 and bo = 1.2 

a cii Social welfare 

Flat rate I L2 21.608 

Marginal cost 1.0131 1.1869 21.6857 
Proposed 1.0308 1.1692 21.7237 

Table 4 Price fluctuation at E = 0.08 and bo = 1.2. The price + Wt equals the marginal cost at stage t. 

p + w pi + toi gi Average (retail) price 

Flat rate 3J)2 YM - 5^971 
Marginal cost 4.324 6.324 - 5.403 
Proposed 4.868 4.505 -2.363 3.568 

For the case where the system operator ramps up the capacity in advance (b = 1.2), and con- 
sumers have a high level of demand substitutability (E = 0.08), the one-stage aggregate demand 
and the social welfare resulting from the three pricing mechanisms are given in Table 3. The prices 
faced by consumers are given in Table 4. From Table 3 we observe that under the proposed pric- 
ing mechanism, consumers would like to shift 0.031 peak load to off-peak hours, while under the 
marginal cost pricing mechanism, consumers are willing to shift less than 0.014 peak load to off- 
peak hours. Compared to marginal cost pricing, the more flattened load curve resulting from the 
proposed pricing mechanism leads to 50% more social welfare gain. 

For a given load curve, the proposed pricing mechanism results in a larger price difference between 
stage 1 and stage than marginal cost pricing, because of the negative price gi. The negative price 
gi creates an additional incentive for consumers to shift their load from stage 1 to stage 0. In this 
way, the proposed pricing mechanism results in a more flattened load curve and a higher social 
welfare than marginal cost pricing (cf. Table 3). 

E.2.2. Peak load reduction. Under flat rate pricing, the peak load (the aggregate demand 
at the second stage) is 1.2, because consumers do not have an incentive to shift their load to off-peak 
hours. Given a pricing mechanism and a system operator's policy (bo), consumers are willing to 
substitute across time only up to a certain level. Even with a high level of demand substitutability, 
consumers prefer not to shift much of their peak load, to avoid the utility loss caused by peak load 
shifting. For example, with b = 1.2 and E = 0.08, consumers under marginal cost pricing choose 
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Figure 4 Comparison of the percentage of peak load reduction (the peak load under flat rate pricing, 1.2, is used 
a reference) resulting from the proposed pricing mechanism and the marginal cost pricing mechanism, 
as a function 6q. 




Time 



Figure 5 Real-time prices and actual system loads on August 01, 2011, ISO New England Inc. Blue bars represent 
the real-time system loads and the dots connected by a black line represent the hourly prices. 



to shift at most 0.013 peak load (cf. Table 3). In Fig. 4, for different values of b , we compare the 
maximum amount of peak load consumers choose to shift under the proposed pricing mechanism 
and the marginal cost pricing mechanism. 

We observe from Fig. 4 that the amount of peak load consumers will shift decreases with b Q . 
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This is because a larger reserve at the first stage lowers the price at the second stage, which in 
turn discourages consumers from shifting their peak load. The proposed pricing mechanism results 
in a peak load which is approximately 1.5 percent lower than that resulting from marginal cost 
pricing, regardless of the value of b . If the system operator ramps up the system capacity in 
advance (b = 1.2), marginal cost pricing reduces the system peak load resulting from flat rate 
pricing by approximately one percent. Compared to marginal cost pricing, the negative price qi in 
the proposed mechanism encourages consumers to make a larger shift of their peak load (cf. the 
discussion at the end of Section E.2.1). 

Fig. 5 plots the real-time system loads and prices on August 1, 2011, a typical hot summer day 
in New England 21 . If consumers are able to shift some of their load to the morning (possibly at 
the expense of losing some utility) , the proposed pricing mechanism encourages consumers to shift 
more of their peak load than marginal cost pricing. Since the highest peak load determines the 
generation capacity necessary for system reliability, the proposed pricing mechanism has a greater 
potential to reduce the long-term capacity investment. 



21 www. fere .go v/market- over sight /mkt- electric/new- england/20 11/08- 20 11- elec-isone-dly.pdf 



